Systems and methods for assisting the hearing-impaired using machine learning for ambient sound analysis and alerts

ABSTRACT

Systems and Methods for assisting the hearing-impaired are described. The methods rely on obtaining audio signals from the ambient environment of a hearing-impaired person. The audio signals are analyzed by a machine learning model that can classify audio signals into audio categories (e.g. Emergency, Animal Sounds) and audio types (e.g. Ambulance Siren, Dog Barking) and notify the user leveraging a mobile or wearable device. The user can configure notification preferences and view historical logs. The machine learning classifier is periodically trained externally based on labelled audio samples. Additional system features include an audio amplification option and a speech to text option for transcribing human speech to text output.

FIELD OF THE DISCLOSURE

The present disclosure generally relates to systems and methods to helpthe hearing-impaired understand ambient sounds in their environment.More particularly, the present disclosure relates to machine learningbased systems that can learn about ambient sounds and assist thehearing-impaired understand them via mobile or embedded device-basednotifications.

BACKGROUND OF THE DISCLOSURE

According to the World Health Organization, 466 million peopleworldwide, or about 6% of the population, including 34 million children,suffer from disabling hearing loss (WHO, 2019). It is estimated that by2050 over 900 million people will have disabling hearing loss. Disablinghearing loss refers to hearing loss greater than 40 decibels.

Individuals in the deaf and hearing loss community have faceddiscrimination and oppression for centuries. This has caused challengesfor them in terms of employment, higher education, and other privilegesthat their hearing counterparts take for granted. They are oftenstereotyped and marginalized in society, and their communicationbarriers have led to a strained relationship with the rest of thecommunity making it difficult for them to live normal daily lives. Astudy published by the British Department of Health suggests thathearing-impaired individuals are 60% more susceptible to mental healthand social anxiety issues than their counterparts with normal hearingabilities (Department of Health, 2005).

Several causes have been identified for hearing loss. The Hearing LossAssociation of America (HLAA) has categorized hearing disabilities intotwo classes: (i) conductive, which include problems associated with theear drum, ear canal, and middle ear function, and (ii) sensorineural,which include issues that affect inner ear functions as well as thebrain and nervous system that interpret auditory inputs (HLAA, 2019).Conductive hearing loss is prompted by ear infections, benign tumors,excessive ear fluid and/or ear wax, or poor function of the ear tubes.As for sensorineural issues, researchers have labeled traumatic events,aging, hereditary, and virus or immune diseases as primary causes.Because of the multitude and diverse range of issues that lead to thisdisability, many individuals are affected and require additionalassistance in their daily lives.

The most popular solution for hearing loss is the hearing aid (NIDCD,2018). Hearing aids are electronic devices generally worn behind orinside the ear. The device is usually battery powered. It receives soundthrough an embedded microphone, which converts the sound waves toelectrical signals that are processed, amplified and played back using aspeaker. The amplifier increases the power of the sound signal thatwould normally reach the ear, allowing the hearing-impaired user tolisten. Hearing aids are primarily useful for people suffering fromsensorineural hearing loss which occurs when some of the small sensorycells in the inner ear, called hair cells, are damaged due to injury,disease, aging, or other causes. Surviving hair cells leverage theamplified sound signal generated by the hearing aid to compensate forthe loss of perception that would occur otherwise and convert thereceived sound signal into impulses sent to the brain via the auditorynerve. However, if the inner ear is too damaged, or the auditory nervehas problems, a hearing aid would be ineffective. The cost of a hearingaid can range from $1,000 to $6,000 (Rains, 2019).

For people suffering from profound hearing loss, cochlear implants maybe an option. Cochlear implants are surgically implantedneuro-prosthetic devices that bypass sensory hair cells used in the earfor normal hearing and attempt to directly stimulate the auditory nervewith electrical signals. With prolonged therapy and training ahearing-impaired person may learn to interpret the signals directly sentto the auditory nerve as sounds and speech. In the US, cochlear implantscan cost approximately $100,000, and for pre-lingually deaf children therisk of not acquiring spoken language even with an implant may be ashigh as 30% (Wikipedia, 2019b).

A variety of assistive technologies have emerged over the years to thehelp the hearing-impaired. These include FM radio-based systems thattransmit radio signals from a speaker to a listener and audio inductionloop systems that pick up electromagnetic signals using a telecoil in ahearing aid, cochlear implant, or headset (Gallaudet, 2019). Mobiledevices with touch screens and real-time text to speech transcriptioncapabilities are starting to get used as well. Closed captioning isbecoming standard in streaming media as well as television programs.Apple recently launched a feature called “Live Listen”, on iOS mobiledevices (e.g. iPhone, iPad) where the device becomes a remote microphoneplaced close to a speaker and a BlueTooth headset replays the sound live(Apple, 2019). This can be useful when you are trying to hear aconversation in a noisy room or for a hearing-impaired student trying tolisten to a teacher across the classroom.

These conventional approaches to assisting the hearing-impaired attemptto create a real-time listening experience similar to a normal person byusing technology to work around the defects of the ear. They can beexpensive and sometimes aesthetically undesirable. Hearing aids arebattery powered and not something a user would like to wear all thetime. There are several situations where a hearing-impaired user mayjust want to be notified about interesting ambient sounds without havingto wear a hearing aid. For example, a user may be sleeping and may wantto get alerted if there is a knock on the door, a baby crying, a smokealarm going off or other similar audio events that warrant some action.A digital assistant that can actively listen, process and notify theuser via a vibration alert on a smart watch can be very useful in thesecircumstances. Hearing-impaired people often get anxious when they arein new surroundings because systems they have in their house (e.g. avisual doorbell or telephone) may not be available in say a hotel.Having a digital assistant that can intelligently process ambient soundsand notify them via their mobile device can be very useful in thesecircumstances and allow the hearing-impaired user to operate moreconfidently. The digital assistant should be customizable such that theuser can specify notification preferences based on audio categories,audio types, time of day, location, etc, and also allow the user theview historical alerts. The assistant can run as an app on a mobiledevice or integrate with smart listening devices such as Amazon Alexaand Google Home that have omnidirectional microphone arrays that canpick-up sounds coming from any direction.

BRIEF SUMMARY OF THE DISCLOSURE

In an exemplary embodiment, a system to assist the hearing-impairedcomprises of an audio receiver communicatively coupled to a processingsystem and a notification system. The processing system obtains audiosignals from the audio receiver and first runs signal processing stepsto reduce background noise and interference. Subsequently, it runs amachine learning based classifier to analyze the audio signal andclassify it into an audio category and audio type. The user is thennotified, based on their preferences, with a summary of the classifiedaudio, and, for the specific type of audio, the user is presented with ameaningful description of what the machine learning classifieridentified the signal as. Notifications can be stored in the system forhistorical viewing. Optionally, the system may include an amplifier andfilter to output the received audio signal to an audio output of theuser's choice or store it as an audio file for future playback. Thesystem can also include a speech to text module that can decipher humanspeech and provide a text transcript of the speech in real-time on theuser's notification screen. The system's machine learning classifier isperiodically trained externally based on labelled audio data and updatedin the system automatically or manually. Methods for training the systemare based on data science principles and a variety of machine learningalgorithms from simple regression to sophisticated deep learning basedneural networks may be utilized based on accuracy, complexity andtraining cost trade-offs.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated and described herein withreference to the various drawings, in which like reference numbers areused to denote like system components/method steps, as appropriate, andin which:

FIG. 1 is a block diagram of the system for assisting hearing-impairedpersons.

FIG. 2 is a high-level flowchart of operation of the system of FIG. 1.

FIG. 3 is a high-level flowchart of the analysis and processing stepsthat capture ambient sound signals and notify the hearing-impaired userbased on preferences.

FIG. 4 is a flowchart of the method used to train the machine learningmodel to classify sounds.

FIG. 5 is a mobile device app showing examples of user configurationoptions.

FIG. 6 is a mobile device app showing examples of user notifications andinteractions.

DETAILED DESCRIPTION OF THE DISCLOSURE

In various embodiments, the present disclosure relates to systems andmethods for assisting the deaf and hearing-impaired. The systems andmethods may use mobile devices or other smart technology (e.g. mobiledevices—iPhone, Android device, tablets, smart watches, etc.) that candetect and process ambient sounds, output information, respond to usersignals (e.g. via audio or touch) and store data sets. These featurescombined helps develop a system where the hearing-impaired can utilizetechnology to inform them of nearby sounds by classifying them intoaudio categories and types. Examples of audio categories include AnimalSounds, Emergency, Devices, Vehicles, Speech, Music, etc. Each audiocategory can have multiple specific audio types, e.g., for the audiocategories listed above, specific audio types could be Dog Barking,Ambulance Siren, Telephone Ring, Garbage Truck, English Conversation,Piano, etc.

FIG. 1 includes a system 100 which comprises of an audio receiver 105connected to a processing system 110 connected to a notification system115. In an example embodiment, the audio receiver 105 could be amicrophone that is coupled with the processing system (e.g. a microphoneon a mobile device or laptop) or an external device like Alexa or GoogleHome that can capture audio samples and send it to the processing systemusing an Application Programming Interface (API). The processing system110 analyzes the received audio signal to detect and classify differentsounds, and based on user preferences, send an appropriate notificationto the user. In an example embodiment, the processing system could be anapp running on a mobile device or a program running on an embeddedsystem (e.g., a Raspberry Pi module). The notification system 115communicates directly with the user based on selected preferences. In anexample embodiment, the notification system could be a push notificationon a mobile device, a vibration alert on a smart watch, or other visualforms of notification using LEDs or display technology.

FIG. 2 illustrates an overall operational flowchart 200 of an examplesystem. The user configures the system 205 based on their preferencesfor what, when, and how they would like to be notified. This may be donethrough user preferences setting on the application software, or in thesettings app of an Apple or Android device or configured with hardwarebuttons on an embedded device. Users can configure their preferencesbased on what sounds or categories of sounds they would like to benotified of (e.g. Animal Sounds, Emergency, Devices, Vehicles, Speech,Music, etc.), and they can also choose how they would like to benotified (e.g. through a text message, vibration alert, or anothermethods discussed in 115). Furthermore, they can decide when they wantto be notified (e.g. at work, at home, or in a public setting) and theycan adjust when they want the system to be active (e.g. some users maywant to use the system when they don't have a hearing aid on them). In210, the audio receiver is configured. This requires that the deviceused to capture the sounds from the environment is activated and able toperform its function. In an example embodiment, if the system is an appon a mobile device, the app must be allowed by the user to access themicrophone on the device. Similarly, if the audio input is coming from adevice like Alexa, the system should have proper API access enabled toget the audio inputs. After the system is configured, in 215, it willwait for a signal to be received based on the set preferences. If nointeresting signal is received, the system will continuously loop backto the audio receiver until appropriate audio input in 220 is found.Once a signal is received, it is processed and analyzed in 225. Thisincludes isolating and classifying the sound, and this process isfurther described by FIG. 3. After the system determines what the soundis, step 230 checks if the detected sound or category matches what wasconfigured in user preferences 205. If a valid notification criterion ismet, the user is notified in step 240, otherwise no notification isgenerated. Note that the system may still log the event although theuser may not have been notified. The system executes the event loop 235endlessly till the application is terminated.

FIG. 3 demonstrates the core processing and analysis steps. In 305 theaudio signal to be processed is isolated from other background noises.Generally, this would mean that a valid audio signal above the noisethreshold has been received. The received signal is run through digitalsignal processing filters that improve the fidelity and quality of thereceived sound, to assist downstream processing and detection. Once thesound has been isolated and filtered, step 310 checks to see if the userwants the specific sound to be amplified and sent to them. If it is, 315will run the amplifier which will increase the audio signal strength andcondition it so that it can be easily perceived by individualsexperiencing hearing loss. In 320, the improved sound will be outputtedto the user through their listening device (e.g. headset). A copy of theaudio signal may also be stored digitally. If the user is completelyvoid of hearing and needs the sound to be communicated to them inanother way, or if the user wants the system to detect and notifyregardless, it will then have to be classified into a specific categoryso that eventually the sound can be identified. Step 325 runs themachine learning classifier which takes the audio signal as input andoutputs a most likely audio category (e.g. Animal Sounds, Emergency,Devices, Vehicles, Speech, Music, etc.) and specific audio type (e.g.Dog Barking, Ambulance Siren, Telephone Ring, Garbage Truck, EnglishConversation, Piano, etc.) that matches a pre-determined set of audiocategories and types that the model has been trained to identify. Onceaudio category and type is determined, step 330 checks whether the usercares to be notified about the detected audio category and type based onpreferences set before. If not, the system goes back to step 305 whereit tries to collect new audio samples for analysis. If the user doeswant to be notified of the sound, the system checks if the determinedcategory was human speech in 340. If so, it proceeds to 345 where itruns the speech to text module which extracts text from the human voicesignal and sends it to the notification system 350. If it is not humanspeech, the audio information is summarized in 355 and the summary issent to the notification system 350. For example, the system may havedetected audio type Ambulance Siren of category Emergency. Thatinformation, along with the date, time duration and other relevantinformation may be sent to the user's notification system (e.g. a mobilephone alert).

FIG. 4 represents the steps used to train the machine learningclassifier 325. In 405, audio data sets including both the training andvalidation sets is imported. A variety of free and commercial audio datasets are available online. For example, Google AudioSet (Gemmeke, 2017)is collection of roughly 2.1 million audio clips, each 10 seconds long,extracted and labelled from YouTube. Similarly, the UrbanSound8K dataset(Salamon, 2014) contains over 8,000 labeled sound files eachapproximately 4 seconds long and of sounds encountered in a typicalurban environment and labelled into 10 classes. The FSD project usescrowdsourcing of annotations of audio samples from Freesound organisedusing the AudioSet framework (Fonseca, 2019). Data sets can also begenerated manually by recording sounds and labeling them. This data isthen normalized in 410. This process includes developing a consistentformat to label and organize audio samples. In 415, features will thenbe extracted from the audio set. For example, differences in frequency,pitch, tone, intensity, etc. can be used to distinguish betweendifferent audio samples. Step 420 selects the model that best classifiesand trains the data set using data science principles. This may be assimple as decision trees, regression or k-nearest neighbors, or it couldbe as advanced as a deep learning neural network. In 425, the model istrained to classify audio samples to appropriate category and type basedon extracted features. A portion of the data set is used for trainingand the rest is used to validate the model in 430. The process is thenrepeated in 435 till an acceptable level of model prediction accuracy isreached, based on precision, accuracy, and recall. Once the model istrained, step 440 deploys it to the system described in FIG. 3. Themodel can periodically be retrained based on the availability of newdata sets or user feedback.

FIG. 5 illustrates an example embodiment of the user's view of thesystem. It includes a sample 500 of the home screen of the user'sinterface, and an example of a settings page 530 that can be integratedinto a mobile application. 505 displays a brief summary of the user'sprofile including a portrait of them and their name. This shows thatthey are logged into their profile and ensures that the application isrunning with their configured preferences. 510 illustrates four mainfunctions of the application—speech to text, amplifier, notifications,and settings. FIG. 6 describes these functions. In 515, the user canclick to view or update their account information. This may includetheir profile, account settings, as well as other important informationthat pertains to their condition so the system can cater to those needs.520 allows the user to logout of their account. This may be useful ifthey want to temporarily disable the system, or if they wish to switchto another user's account with different settings (e.g. a shared devicein school). 530 exemplifies a sample of the settings page of the user'ssystem. 535 allows them to choose whether they want to allownotifications to be outputted on their device. If the switch is on,notifications will be sent to the user each time one of their preferredsounds is detected. If it is off, no notifications will be sent to theuser although data will continue to be collected and stored. 540exhibits sample categories the user may prefer to be notified of. Forexample, if they wish to be notified of any Animal Sounds, they choosethat Animal Sounds category and specify specific audio types (e.g. DogBarking) or all audio types in the category. The system could alsoprovide feedback to the application developer to improve model trainingbased on frequently requested categories.

FIG. 6 is an example embodiment of the user's interactions with thesystem. 600 represents one of the functions of the system—speech to textconversion. This can be used to convert human speech into text. 605 isthe menu bar that can be found on each page. By pressing the threehorizontal bars, the user will be returned to the home screen 500.Similarly, the search icon after it can be used to search for anythingin the system, and the three dots allows the user to customize specificsettings for each page (e.g. language, volume, etc.). If the microphoneicon 610 is pressed, the message “Listening . . . ” will be presented toshow the system is activated. It will continue to show this messageuntil 610 is pressed again to stop transcribing the audio signal. As thespeech is being captured, its text will be transcribed in real-time inthe text-box 615. 620 shows the amplifier feature of the system. Similarto 610, when the speaker icon 625 is selected, it will activate theamplifier. The user can adjust what volume they want the sound to beamplified to using the slider 630. While the amplifier is running, themessage 635, “Playing on Device . . . ” or another similar message willbe outputted so the user knows that it is properly functioning. When thesound is amplified and is being played back, 640 depicts an animation ofthe time domain samples of the sound waves as it is being played back.650 is a sample of the notifications page of the application. 655 allowsthe user to view past notifications from the last 24 hours, or from thelast 7 days, for example. 660 is an example embodiment of notificationsthat might be sent to the user. It includes the category of thenotification (e.g. Animal Sounds, Emergency, Devices, Vehicles, Speech,Music, etc.), specific audio type (e.g. Dog Barking, Ambulance Siren,Telephone Ring, Garbage Truck, English Conversation, Piano, etc.) andtime when the alert was generated.

It will be appreciated that some embodiments described herein mayinclude or utilize one or more generic or specialized processors (“oneor more processors”) such as microprocessors; Central Processing Units(CPUs); Digital Signal Processors (DSPs): customized processors such asNetwork Processors (NPs) or Network Processing Units (NPUs), GraphicsProcessing Units (GPUs), or the like; Field-Programmable Gate Arrays(FPGAs); and the like along with unique stored program instructions(including both software and firmware) for control thereof to implement,in conjunction with certain non-processor circuits, some, most, or allof the functions of the methods and/or systems described herein.Alternatively, some or all functions may be implemented by a statemachine that has no stored program instructions, or in one or moreApplication-Specific Integrated Circuits (ASICs), in which each functionor some combinations of certain of the functions are implemented ascustom logic or circuitry. Of course, a combination of the approachesmay be used. For some of the embodiments described herein, acorresponding device in hardware and optionally with software, firmware,and a combination thereof can be referred to as “circuitry configuredto,” “logic configured to,” etc. perform a set of operations, steps,methods, processes, algorithms, functions, techniques, etc. on digitaland/or analog signals as described herein for the various embodiments.

Moreover, some embodiments may include a non-transitorycomputer-readable medium having instructions stored thereon forprogramming a computer, server, appliance, device, processor, circuit,etc. to perform functions as described and claimed herein. Examples ofsuch non-transitory computer-readable medium include, but are notlimited to, a hard disk, an optical storage device, a magnetic storagedevice, a Read-Only Memory (ROM), a Programmable ROM (PROM), an ErasablePROM (EPROM), an Electrically EPROM (EEPROM), Flash memory, and thelike. When stored in the non-transitory computer-readable medium,software can include instructions executable by a processor or device(e.g., any type of programmable circuitry or logic) that, in response tosuch execution, cause a processor or the device to perform a set ofoperations, steps, methods, processes, algorithms, functions,techniques, etc. as described herein for the various embodiments.

Although the present disclosure has been illustrated and describedherein with reference to preferred embodiments and specific examplesthereof, it will be readily apparent to those of ordinary skill in theart that other embodiments and examples may perform similar functionsand/or achieve like results. All such equivalent embodiments andexamples are within the spirit and scope of the present disclosure, arecontemplated thereby, and are intended to be covered by the followingclaims.

REFERENCES

-   World Health Organization: WHO. (2019, March 20). Deafness and    hearing loss.    https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss-   WebMD. (2012, May 14). Treatments for Hearing Loss.    https://www.webmd.com/a-to-z-guides/hearing-loss-treatment-options-   National Institute on Deafness and Other Communication Disorders:    NIDCD. (2019, November 12). Assistive Devices for People with    Hearing, Voice, Speech, or Language.    https://www.nidcd.nih.gov/health/assistive-devices-people-hearing-voice-speech-or-language-disorders-   Department of Health (2005). Mental health and deafness—Towards    equity and access: Best practice guidance. London, UK: HMSO-   Hearing Loss Association of America: HLAA. (2019). Types, Causes and    Treatments,    https://www.hearingloss.org/hearing-help/hearing-loss-basics/ypes-causes-and-treatment/-   National Institute on Deafness and Other Communication Disorders:    NIDCD. (2018, June 15). Hearing Aids.    https://www.nidcd.nih.gov/health/hearing-aids-   Rains, T. (2019, September 13). How much do hearing aids    cost?https://www.consumeraffairs.com/health/hearing-aid-cost.html-   Wikipedia. (2019b, November 24). Cochlear implant.    https://en.wikipedia.org/wiki/Cochlear_implant-   Gallaudet University and Clerc Center. (2019). Assistive    Technologies for Individuals Who are Deaf or Hard of Hearing.    https://www3.gallaudet.edu/clerc-center/info-to-go/assistive-technology/assistive-technologies.html-   Apple. (2019, September 19). Use Live Listen with Made for iPhone    hearing aids. https://support.apple.com/en-us/HT203990-   Gemmeke, J. (2017). Audio Set: An ontology and human-labeled dataset    for audio events. https://research.google.com/audioset/-   Salamon, J. (2014). A Dataset and Taxonomy for Urban Sound Research.    https://urbansounddataset.weebly.com/-   Fonseca, E. (2019). Freesound Datasets: A Platform for the Creation    of Open Audio Datasets. https://annotator.freesound.org/fsd/explore/

What is claimed is:
 1. A system comprising: an audio receiver; aprocessing system connected to the audio receiver; a notification systemconnected to the processing system, wherein the processing system isconfigured to i) obtain audio signal from the audio receiver; ii)process the audio signal to reduce noise and interference and check ifthe audio signal contains appropriate audio; iii) responsive to theaudio signal containing appropriate audio, run a machine learning basedclassifier to analyze the audio signal, otherwise loop back to i); iv)classify the audio signal into an audio category and audio type based onthe machine learning based classifier, wherein the audio categoryincludes one of animal sounds, emergency sounds, device sounds, vehiclesounds, speech, and music, and wherein the audio type is one of aplurality of types specific to each audio category; v) notify a user viathe notification system of the detected audio category and type;wherein, for the notification, the user is presented with textassociated with the classified audio, and, for the specific type ofaudio, the user is presented with a meaningful description of what themachine learning process characterized the isolated signals as thatincludes the audio type and additional relevant information, wherein,when the audio category is speech, the meaningful description includestext that corresponds to the speech based on a conversion; and vi) loopback to i).
 2. The system of claim 1, wherein the processing system hasa filter and an amplifier to output an improved copy of the receivedaudio signal to a user's hearing device or store it digitally.
 3. Thesystem of claim 1, wherein the notification system is a mobile devicepush notification configured by the user.
 4. The system of claim 1,wherein the notification system is a wearable device that can generatevibration alerts and display information on a digital screen.
 5. Thesystem of claim 1, wherein the notification preferences can beconfigured by the user based on audio category and audio type.
 6. Thesystem of claim 1, wherein the machine learning classifier isperiodically trained externally based on labelled audio sample data andupdated in the system.
 7. The system of claim 6, where the machinelearning training system is further configured to receive feedback fromthe user that the detected audio category and type were incorrect orunknown, and process the feedback for the labelled audio sample dataincluding a new audio category and a new audio type when the feedbackidentifies the new audio category and the new audio type.
 8. The systemof claim 1, where the entire system is running as an application on amobile phone, wherein the audio receiver is the microphone on the mobiledevice, the processing system is the CPU on the mobile device and thenotification system is the screen and vibration alerts.
 9. The system ofclaim 1, wherein the audio receiver is a separate device communicativelycoupled to the processing system running on mobile device.
 10. A methodcomprising: i) obtaining audio signal from the audio receiver; ii)processing the audio signal to reduce noise and interference andchecking if the audio signal contains appropriate audio; iii) responsiveto the audio signal containing appropriate audio, running a machinelearning based classifier to analyze the audio signal, otherwise loopingback to i); iv) classifying the audio signal into an audio category andaudio type, wherein the audio category includes one of animal sounds,emergency sounds, device sounds, vehicle sounds, speech, and music, andwherein the audio type is one of a plurality of types specific to eachaudio category; v) notifying a user via the notification system of thedetected audio category and type; wherein, for the notification, theuser is presented with text associated with the classified audio, and,for the specific type of audio, the user is presented with a meaningfuldescription of what the machine learning process characterized theisolated signals as that includes the audio type and additional relevantinformation, wherein, when the audio category is speech, the meaningfuldescription includes text that corresponds to the speech based on aconversion; and vi) looping back to i).
 11. The method of claim 10,further comprising of an amplifier and filter to output an improved copyof the received audio signal to a user's hearing device or store itdigitally.
 12. The method of claim 10, wherein the notification methodis a mobile device push notification.
 13. The method of claim 10,wherein the notification method uses a wearable device that can generatevibration alerts and display information on a digital screen.
 14. Themethod of claim 10, wherein the notification preferences can beconfigured by the user based on audio category and audio type.
 15. Themethod of claim 10, wherein the machine learning classifier isperiodically trained externally based on labelled audio sample data andupdated.
 16. The method of claim 10, where the machine learning trainingincludes steps to receive feedback from the user that the detected audiocategory and type were incorrect or unknown, and process the feedbackfor the labelled audio sample data including a new audio category and anew audio type when the feedback identifies the new audio category andthe new audio type.
 17. A non-transitory computer-readable mediumcomprising instructions that, when executed, cause a processing systemto perform the steps of: i) obtaining audio signal from the audioreceiver; ii) processing the audio signal to reduce noise andinterference and checking if the audio signal contains appropriateaudio; iii) responsive to the audio signal containing appropriate audio,running a machine learning based classifier to analyze the audio signal,otherwise looping back to i); iv) classifying the audio signal into anaudio category and audio type, wherein the audio category includes oneof animal sounds, emergency sounds, device sounds, vehicle sounds,speech, and music, and wherein the audio type is one of a plurality oftypes specific to each audio category; v) notifying a user via thenotification system of the detected audio category and type; wherein,for the notification, the user is presented with text associated withthe classified audio, and, for the specific type of audio, the user ispresented with a meaningful description of what the machine learningprocess characterized the isolated signals as that includes the audiotype, wherein, when the audio category is speech and additional relevantinformation, the meaningful description includes text that correspondsto the speech based on a coversion; and vi) looping back to i).